LWSIS: LiDAR-Guided Weakly Supervised Instance Segmentation for Autonomous Driving

Image instance segmentation is a fundamental research topic in autonomous driving, which crucial for scene understanding and road safety. Advanced learning-based approaches often rely on the costly 2D mask annotations training. In this paper, we present more artful framework, LiDAR-guided Weakly Supervised Instance Segmentation (LWSIS), leverages off-the-shelf 3D data, i.e., Point Cloud, together with boxes, as natural weak supervisions training image models. Our LWSIS not only exploits complementary information multimodal data during but also significantly reduces annotation cost of dense masks. detail, consists two modules, Label Assignment (PLA) Graph-based Consistency Regularization (GCR). The former module aims to automatically assign point cloud point-wise labels, while atter further refines predictions by enforcing geometry appearance consistency data. Moreover, conduct secondary nuScenes, named nuInsSeg, encourage perception tasks. Extensive experiments well large-scale Waymo, show that can substantially improve existing weakly supervised models involving Additionally, be incorporated into object detectors like PointPainting boost detection performance free. code dataset are available at https://github.com/Serenos/LWSIS.